process at any point, thereby obtaining 
the best possible result for a given 
amount of computation time. Alterna- 
tively, the results can be displayed as 
they are generated, providing the user 
with real-time feedback about the cur- 
rent accuracy of classification. 

Computational savings are realized 
through the guided application of re- 
sources only to those items that are es- 
timated to be misclassified. The coarse 
approximation may suffice for items 


that can be classified easily, and more 
computation can be devoted to am- 
biguous or difficult cases. Thus, the al- 
gorithm enables the user to exert di- 
rect, dynamic control over the balance 
between classification speed and accu- 
racy. When constraints on computation 
time and other resources preclude a to- 
tally accurate classification of all the 
data, this algorithm provides the best 
possible approximation to the classifi- 
cation of each item, rather than fully 


classifying only a fraction of the data 
set and leaving the rest marked “un- 
known.” 

This work was done by Kiri Wagstaff and 
Michael Kocurek of Caltech for NASA’s fet 
Propulsion Laboratory. Further information is 
contained in a TSP (see page 1). 

The software used in this innovation is 
available for commercial licensing. Please 
contact Karina Edmonds of the California In- 
stitute of Technology at (626) 395-2322. 
Refer to NPO-44089. 


Active Learning With Irrelevant Examples 

Classification algorithms can be trained to recognize and reject irrelevant data. 

NASA’s Jet Propulsion Laboratory, Pasadena, California 


An improved active learning method 
has been devised for training data classi- 
fiers. One example of a data classifier is 
the algorithm used by the United States 
Postal Service since the 1960s to recog- 
nize scans of handwritten digits for pro- 
cessing zip codes. Active learning algo- 
rithms enable rapid training with 
minimal investment of time on the part 
of human experts to provide training ex- 
amples consisting of correctly classified 
(labeled) input data. They function by 
identifying which examples would be 
most profitable for a human expert to 
label. The goal is to maximize classifier 
accuracy while minimizing the number 
of examples the expert must label. 

Although there are several well-estab- 
lished methods for active learning, they 
may not operate well when irrelevant ex- 
amples are present in the data set. That 


is, they may select an item for labeling 
that the expert simply cannot assign to 
any of the valid classes. In the context of 
classifying handwritten digits, the irrele- 
vant items may include stray marks, 
smudges, and mis-scans. Querying the 
expert about these items results in 
wasted time or erroneous labels, if the 
expert is forced to assign the item to one 
of the valid classes. 

In contrast, the new algorithm pro- 
vides a specific mechanism for avoiding 
querying the irrelevant items. This algo- 
rithm has two components: an active 
learner (which could be a conventional 
active learning algorithm) and a rele- 
vance classifier. The combination of 
these components yields a method, de- 
noted Relevance Bias, that enables the 
active learner to avoid querying irrele- 
vant data so as to increase its learning 


rate and efficiency when irrelevant items 
are present. 

The algorithm collects irrelevant data 
in a set of rejected examples, then trains 
the relevance classifier to distinguish be- 
tween labeled (relevant) training exam- 
ples and the rejected ones. The active 
learner combines its ranking of the items 
with the probability that they are relevant 
to yield a final decision about which item 
to present to the expert for labeling. Ex- 
periments on several data sets have 
demonstrated that the Relevance Bias 
approach significantly decreases the 
number of irrelevant items queried and 
also accelerates learning speed. 

This work was done by Kiri Wagstaff of Cal- 
tech and Dominic Mazzoni of Google, Inc. for 
NASA’s Jet Propulsion Laboratory. For more in- 
formation, contact iaoffice@jpl.nasa.gov. 
NPO-44094 


© A Data Matrix Method for Improving the Quantification of 
Element Percentages of SEM/EDX Analysis 

John F. Kennedy Space Center, Florida 


A simple 2D MxN matrix involving 
sample preparation enables the micro- 
analyst to peer below the noise floor of 
element percentages reported by the 
SEM/EDX (scanning electron mi- 
croscopy/ energy dispersive x-ray) 
analysis, thus yielding more meaning- 
ful data. 

Using the example of a 2x3 sample 
set, there are M = 2 concentration levels 


of the original mix under test: 10 per- 
cent ilmenite (90 percent silica) and 20 
percent ilmenite (80 percent silica) . For 
each of these M samples, N= 3 separate 
SEM/EDX samples were drawn. In this 
test, ilmenite is the element of interest. 
By plotting the linear trend of the M 
sample’s known concentration versus 
the average of the N samples, a much 
higher resolution of elemental analysis 


can be performed. The resulting trend 
also shows how the noise is affecting the 
data, and at what point (of smaller con- 
centrations) is it impractical to try to ex- 
tract any further useful data. 

This work was done by fohn Lane of 
Kennedy Space Center. For further informa- 
tion, contact the Kennedy Innovative Part- 
nerships Program Office at (321) 861-7158. 
KSC-13303 
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